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ABSTRACT 

As is consistent with national trends, the Montgomery 
County (Maryland) Public School System is exploring the use of 
instruments other than multiple-choice tests for high-stakes testing. 
This paper presents information on racial, ethnic, and gender 
differences in performance on the various types of tests being 
administered in the district. Sharing such information among school 
systems will help in the evaluation of new types of assessment. The 
six assessments used in the study were: (1) a mathematics multiple 

choice test given to grades 3 to 8; (2) a mathematics short answer 
test for grades 3 to 8; (3) a locally developed mathematics extended 
answer test for grades 4, 6, and 7; (4) a reading multiple choice 

test for grades 3 to 8; (5) a language arts extended answer test for 

grades 4, 6, and 7; and (6) the Maryland School Performance 
Assessment program for grades 3, 5, and 8. There were no meaningful 
differences in mathematics performance by racial and ethnic group 
across the different types of test studied. Nonmultiple-choice 
reading and language arts assessments favored nonwhite students. 
Nonmultiple-choice tests, whether in mathematics or language arts and 
reading, favored females over males. The largest difference between 
students on reduced-cost or free meals and others was in reading and 
language arts, where lower income students had subs tant ial ly larger 
gains when moving from multiple-choice to nonmultiple-choice 
assessments. (Contains one reference and three tables.) (SLD) 
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Performance on Different Test Types 
by Racial\Ethnic Group and Gender 



"Machine-scorable , group administered, standardized tests 
are widely used to monitor and report student achievement 
to the public, to inform broad educational policy, and as 
a mechanism for school and teacher accountability." (Bond, 

1995) 

This statement has applied to standardized tests in the Montgomery 
County Public Schools (MCPS) for at least three decades. Starting in 
the late 1970 's this use has included breakdowns of the results by 
racial\ethnic group and by gender. Review of test reports from other 
districts and conversations with district testing and research 
directors indicates that we are not alone in using test results in 
this way. 

Consistent with the movement across the country, MCPS is exploring 
the use of non-multiple choice instruments for high stakes testing. 

As we begin to get data from these assessments one of the issues we 
are addressing is the effect of different modes of assessment on the 
reported group differences. If the differences change, or even if 
they remain the same, is that real or is it a function of the new 
instruments? One way for districts like ours to evaluate this effect 
is to see what is happening in other districts. This paper has two 
purposes — 1) to present racial\ethnic and gender data comparing 
performance on various types of assessments, and 2) to promote the 
sharing of such data among districts. While many variables need to 
be considered when comparing such data across districts, the first 
step is simply to make the data available. 



Description of the Assessments and Population 

The six assessments used in this study are described below. 

Math Multiple Choice (MMC) was administered in Grades 3 to 8 
and was locally developed. The number of questions varied by 
grade from 33 to 43. 

Math Short Answer (MSA) was administered in Grades 3 to 8 . The 
instruments used in Grades 4 and 6 were the Mathematics Goals 
Tests developed by the Psychological Corporation. The 

instruments used in the other grades were locally developed. 
All of these instruments had 10 questions, each scored from 0 
to 3. The total score (maximum 30 points) is used in the 
analysis . 

Math Extended Answer (MEA) was administered in Grades 4, 6, and 
7 and was locally developed. The instrument was one multi-step 
activity scored holistically from 0 to 6 . 



Reading Multiple Choice (RMC) was administered in Grades 3 to 
8. It is the short version of the Reading Comprehension 
section of the Metropolitan Achievement Test, 7th Edition, 
published by the Psychological Corporation. It contains 30 
questions related to six reading passages in each grade. 

Language Arts Extended Answer (LEA) was administered in Grades 
4, 6, and 7. It is part of the Language Arts Performance 
Assessment series published by the Psychological Corporation. 

At each grade there is one reading and writing activity that is 
scored from 0 to 4 on each of three domains. For this study, 
the domain scores will be added to get a total score (maximum 
12 points) to simplify the analysis. 

Maryland School Performance Assessment Program (MSP) was 
administered in Grades 3, 5, and 8. It was developed by the 
state of Maryland and measures six subject areas. Only the 
mathematics, reading, and writing scores will be used in this 
study. The test is a combination of short and extended answer 
items with a 3-digit scale score being reported for each 
subject. 

All of these tests were administered in the spring of 1995 in all 
schools in the district. 

The Montgomery County Public Schools are a large suburban school 
district that has been near the upper end of the socio-economic 
scale. However, in recent years the population has started becoming 
more diverse. 

Data Analysis 



Each instrument used in this study had a different mean and standard 
deviation. These statistics are presented in Table 1 with the 
correlations between the instruments. The different statistics for 
each instrument made it necessary to standardi 2 e the scores to be 
able to compare group differences. This was done by expressing each 
student's score on each test in standard deviation units ( 2 -scores). 
The transformation for each score, X, was done with the following 
formula. 



2 = (X - Mean) \Standard Deviation 

This created score distributions for each instrument that had a mean 
of 0 and a standard deviation of 1. 

Racial group test score differences are often found to be related to 
socio-economic variables. The only such variable that was available 
for the students in this study was participation in the federal Free 
or Reduced Meals ( FRMS) program. The effect of this participation on 
the assessment results being studied is included in this paper. 
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Results 



The mean z-scores for each racial\ethnic group and gender are 
presented in Table 2. There are two sets of data in this table 

because different grades took different sets of tests. Similar data 
are presented in Table 3 broken out by students who did and did not 
participate in the FRMS program. For the analyses discussed below, 
the mean scores presented in Tables 2 and 3 have been averaged across 
grade levels to minimize the possible chance effect of one test at 
one grade level. These summary averages are presented in tables 
headed "Table 2 Summary" and "Table 3 Summary" . 

Math — There were no meaningful differences in math performance by 
racial\ethnic group across the different types of tests studied. The 
largest difference in average z-score for a group was for Asian 

Americans whose MSP score was .14 points lower than their multiple 
choice score. This was also the group and instruments that had the 
largest difference between students in FRMS and not in FRMS. The 

Asian Americans in FRMS had a .28 point decline going from multiple 

choice to MSP while the group not in FRMS had only a .11 decline. 
All the other differences between FRMS and non- FRMS groups were .10 
or less. 

The largest difference in scores by gender was .08 from multiple 
choice to MSP. While this difference was not especially large, it 
should be noted that, for all non-multiple choice instruments, scores 
for females increased over their multiple choice scores while scores 
for males were lower on non-multiple choice assessments. 

Reading\Language Arts — The score differences from multiple choice 
to non-multiple choice reading\language arts displayed a clear trend. 
In all cases, the average z-scores for non-White students increased 
while the averages for White students decreased. This meant the 
differences between the non-White and White groups all became 
smaller. To put these trends in perspective, it should be noted that 
the largest difference was from the multiple choice to MSP writing 
where the average for African Americans went up .19 points and the 
average for Whites went down .09 points. Thus, the largest change 
in group differences was between one-quarter and one-third of a 
standard deviation. 

The trends showing declines in the White\non-White differences were 
consistent for both FRMS and non-FRMS students. However, the trends 
toward higher non-multiple choice scores were stronger for the FRMS 
group where there was an overall .23 point gain from multiple choice 
to MSP writing. This was led by a .31 difference for African 
American students. 

The largest differences in reading\language arts scores by gender 
were .10 from multiple choice to MSP writing and reading extended 
answer. In all cases the average score increased for females and 
decreased for males when going from multiple choice to some other 
format . 
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Summary and Discussion 

There were three trends of note in the data presented in this paper. 

Non-multiple choice reading\language arts (R\LA) assessments 
favored non-White students. All three non-White groups in the 
study had higher scores on the non-multiple choice R\LA 
instruments than on the multiple choice. White students had 
lower non-multiple choice scores. This trend of non-multiple 
choice R\LA tests favoring non-White students was the same for 
students participating in FRMS and those not participating. 

Non-multiple choice tests favored females to males. For all 
math and R\LA assessments, females scored higher on the 
non-multiple choice versions than on the multiple choice. For 
males the trend was reversed. 

The largest difference between FRMS and non-FRMS students was 
in R\LA where the FRMS students had substantially larger gains 
when moving from multiple choice to non-multiple choice 
assessments. 

The limited number of instruments reported here as well as the fact 
that students from only one school district were included clearly 
means that the results are not general izable. In fact, the trends 
for the R\LA tests are not consistent with those reported for the 
National Assessment of Educational Progress (Bond, 1995). We need 
similar reports on other tests, different populations, and additional 
socio-economic variables. 
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Table 1 

Statistics for the Tests Used to Compare Racial/Ethnic Groups and Gender Results 



Test 


Mean 


Standard 

Deviation 


MSA 


Correlations 
MEA RMC 


REA 


Grade 4 (N=8007) 














Math Multiple Choice (MMC) 


28.69 


6.44 


.76 


.73 


.66 


.61 


Math Short Answer (MSA) 


19.74 


6.51 




.72 


.65 


.60 


Math Extended Answer (MEA) 


3.66 


1.51 






.61 


.56 


Reading Multiple Choice (RMC) 


21.67 


6.02 








.62 


Reading Extended Answer (REA) 


8.80 


2.15 










Grade 6 (N=7367) 














Math Multiple Choice (MMC) 


30.77 


7.95 


.79 


.72 


.66 


.56 


Math Short Answer (MSA) 


20.94 


6.64 




.70 


.66 


.56 


Math Extended Answer (MEA) 


3.48 


1.70 






.60 


.51 


Reading Multiple Choice (RMC) 


21.80 


5.22 








.56 


Reading Extended Answer (REA) 


7.43 


2.20 










Grade 7 (N=7328) 














Math Multiple Choice (MMC) 


27.46 


7.72 


.82 


.74 


.66 


.58 


Math Short Answer (MSA) 


16.57 


8.16 




.76 


.65 


.59 


Math Extended Answer (MEA) 


2.96 


1.61 






.58 


.53 


Reading Multiple Choice (RMC) 


22.01 


5.62 








.57 


Reading Extended Answer (REA) 


7.74 


2.21 











Test 


Mean 








Correlations 






Standard 

Deviation 


MSA 


MSP Math 


RMC 


MSP 

Reading 


MSP 

Writing 


Grade 3 (N=7352) 
















Math Multiple Choice (MMC) 


26.68 


6.45 


.80 


.67 


.65 


.55 


.43 


Math Short Answer (MSA) 


19.45 


6.84 




.72 


.68 


.59 


.47 


MSP Math 


535.80 


45.21 






.65 


.63 


.49 


Reading Multiple Choice (RMC) 


19.50 


6.40 








.62 


.47 


MSP Reading 


522.19 


42.50 










.51 


MSP Writing 


526.11 


50.09 












Grade 5 (N=7049) 
















Math Multiple Choice (MMC) 


25.31 


7.13 


.84 


.71 


.64 


.52 


.45 


Math Short Answer (MSA) 


18.59 


7.80 




.73 


.66 


.55 


.48 


MSP Math 


530.72 


48.91 






.62 


.58 


.51 


Reading Multiple Choice (RMC) 


22.70 


5.47 








.55 


.47 


MSP Reading 


517.52 


40.47 










.52 


MSP Writing 


513.34 


55.23 












Grade 8 (N=5705) 














.50 


Math Multiple Choice (MMC) 


23.01 


6.34 


.82 


.72 


.66 


.54 


Math Short Answer (MSA) 


14.97 


6.99 




.72 


.65 


.56 


.53 


MSP Math 


535.59 


46.15 






.66 


.61 


.56 


Reading Multiple Choice (RMC) 


22.28 


5.99 








.58 


.51 


MSP Reading 


521.27 


31.58 










.63 


MSP Writing 


506.81 


49.38 
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AVERAGE STANDARDIZED SCORES BY RACIAL/ETHNIC GROUP AND GENDER 

GRADES 3. 5. AND 8 
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GRADES 4. 6. AND 7 
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